AMENDMENT TO THE CLAIMS 

This listing of claims will replace all prior versions and listings of claims in this 
application. 

1 . (Previously Presented) A method for evaluating group-administered test results, 
said method comprising: 

(a) providing a set of class profiles by test administrator, each class profile 
indicative of the question by question performance of a respective class on a selected group- 
administered test comprising a plurality of questions which are directed to measure the same 
characteristic; and 

(b) generating with a computer a normative profile indicative of normative 
class performance on the selected test, where the members of each class included in the 
normative profile are subject to the direction of the same test administrator on the selected test; 

(c) comparing each of the class profiles with the normative profile on a 
question-by-question basis; and 

(d) generating a measure of the question by question variation of eash of the 
class profiles from the normative profile. 

2. (Previously Presented) A method for evaluating group-administered test results, 
said method comprising: 

(a) providing a set of class profiles by test administrator, each class profile 
indicative of the question by question performance of a respective class on a selected group- 
administered test comprising a plurality of questions which are directed to measure the same 
characteristic; 




(b) generating with a computer a normative profile indicative of normative 
class performance on the selected test, where the members of each class included in the 
normative profile are subject to the direction of the same test administrator on the selected test; 

(c) generating a measure of the question by question variation of each of the 
class profiles from the normative profile; and 

(d) identifying individual ones of the class profiles that deviate excessively 
from the normative profile on a question-by-question basis. 

3. (Currently Amended) The method of Claim 1 erf-wherein act (c) comprises 
identifying selected ones of the class profiles that are characterized by a low correlation with the 
normative profile. 

4. (Currently Amended) The method of Claim 1 e?3r wherein act (c) comprises 
comparing (1) at least two portions of each class profile that differ in question difficulty against 
(2) corresponding portions of the normative profile. 

5. (Currently Amended) The method of Claim 1 erf- wherein act (c) comprises 
comparing (1) a difference between at least a portion of each of the class profiles and a 
corresponding portion of the normative profile on a question-by-question basis; and (2) a 
measure of expected variability of class performance for the corresponding portion of the class 
profiles. 

6. (Currently Amended) The method of Claim 1 ef-2r- wherein act (c) comprises 
comparing (1) a measure of difference between a first portion of each of the class profiles and a 
corresponding first portion of the normative profile and (2) a measure of difference between a 



3 




second portion of each of the class profiles and a corresponding second portion of the normative 
profile. 

7. (Currently Amended) The method of Claim 1 efi- wherein act (c) comprises 
comparing the frequency of successive equal measures of class success in answering the 
questions on a question-by-question basis. 

8. (Original) The method of Claim 1 or 2 wherein act (a) comprises: 

(al) providing a respective individual profile for each of a plurality of 
individuals that took the group-administered test, each individual associated with a respective 
one of the classes, each individual profile indicative of accuracy of the respective individual in 
answering the plurality of questions on a question-by-question basis; and 

(a2) determining each class profile from the individual profiles of the 
individuals in the respective class, each class profile indicative of accuracy of the individuals in 
the respective class in answering the questions on a question-by-question basis. 

9. (Original) The method of Claim 1 or 2 wherein the class profiles of (a) are each 
characterized by a skill level, and wherein the normative profile of (b) is indicative of expected 
class performance for a class of said skill level. 

10. (Original) The method of Claim 1 or 2 wherein the normative profile of (b) is 
indicative of expected class performance for classes of a wide range of skill levels. 

1 1 . (Previously Presented) A method for evaluating standardized test results, said 
method comprising: 
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(a) providing a set of sub-group profiles, each sub-group comprising a 
plurality of individuals whose test question responses are exposed to a common element affecting 
test administration and each sub-group profile indicative of the question by question performance 
of a respective sub-group on a selected standardized test comprising a plurality of questions 
which are directed to measure the same characteristic; 

(b) generating with a computer a normative profile indicative of normative 
sub-group performance on the selected test; and 

(c) evaluating administration of the selected test based on a comparison of 
each of the sub-group profiles with the normative profile on a question-by-question basis. 

12. (Original) The method of Claim 1 1 wherein each sub-group comprises 
individuals to whom the selected test was administered by a respective test administrator in a 
respective testing session. 

13. (Original) The method of Claim 1 1 or 12 wherein act (c) comprises identifying 
selected ones of the sub-group profiles that are characterized by a low correlation with the 
normative profile. 

14. (Original) The method of Claim 1 1 or 12 wherein act (c) comprises comparing 

(1) at least two portions of each sub-group profile that differ in question difficulty against 

(2) corresponding portions of the normative profile. 

15. (Original) The method of Claim 1 1 or 12 wherein act (c) comprises comparing 
(1) a difference between at least a portion of each of the sub-group profiles and a corresponding 
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portion of the normative profile on a question-by-question basis; and (2) a measure of expected 
variability of sub-group performance for the corresponding portion of the sub-group profiles. 

16. (Original) The method of Claim 1 1 or 12 wherein act (c) comprises comparing 
(1) a measure of difference between a first portion of each of the sub-group profiles and a 
corresponding first portion of the normative profile and (2) a measure of difference between a 
second portion of each of the sub-group profiles and a corresponding second portion of the 
normative profile. 

17. (Previously Presented) The method of Claim 11 or 12 wherein act (c) comprises 
comparing the frequency of successive equal measures of class success in answering the 
questions on a question-by-question basis. 

18. (Original) The method of Claim 1 1 or 12 wherein act (a) comprises: 
(al) providing a respective individual profile for each of a plurality of 

individuals that took the standardized test, each individual associated with a respective one of the 
sub-groups, each individual profile indicative of accuracy of the respective individual in 
answering the plurality of questions on a question-by-question basis; and 

(a2) determining each sub-group profile from the individual profiles of the 
individuals in the respective sub-group, each sub-group profile indicative of accuracy of the 
individuals in the respective sub-group in answering the questions on a question-by-question 
basis. 
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19. (Original) The method of Claim 1 1 or 12 wherein the sub-group profiles of 

(a) are each characterized by a skill level, and wherein the normative profile of (b) is indicative 
of expected sub-group performance for a sub-group of said skill level. 

20. (Original) The method of Claim 1 1 or 12 wherein the normative profile of (b) is 
indicative of expected sub-group performance for sub-groups of a wide range of skill levels. 

21 . (New) The method of Claim 2 wherein act (d) comprises identifying selected 
ones of the class profiles that are characterized by a low correlation with the normative profile. 

22. (New) The method of Claim 2 wherein act (d) comprises comparing (1) at least 
two portions of each class profile that differ in question difficulty against (2) corresponding 
portions of the normative profile. 

23. (New) The method of Claim 2 wherein act (d) comprises comparing (1) a 
difference between at least a portion of each of the class profiles and a corresponding portion of 
the normative profile on a question-by-question basis; and (2) a measure of expected variability 
of class performance for the corresponding portion of the class profiles. 

24. (New) The method of Claim 2 wherein act (d) comprises comparing (1) a 
measure of difference between a first portion of each of the class profiles and a corresponding 
first portion of the normative profile and (2) a measure of difference between a second portion of 
each of the class profiles and a corresponding second portion of the normative profile. 
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25. (New) The method of Claim 2 wherein act (d) comprises comparing the 
frequency of successive equal measures of class success in answering the questions on a 
question-by-question basis. 
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